Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix nit in LoRA doc #1054

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix nit in LoRA doc #1054

wants to merge 1 commit into from

Conversation

awni
Copy link
Member

@awni awni commented Oct 16, 2024

Very tiny fix, closes #1053

@madroidmaq
Copy link
Contributor

madroidmaq commented Oct 17, 2024

hi @awni , According to OpenAI's official documentation, this may not be a bug, as arguments is not an object but a "string" type "object" that needs to be deserialized into a real object.

For details, see: https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples

image

@hansvdam
Copy link

hansvdam commented Oct 17, 2024

Yes it is true that it should be like that according to the openai docs, but MLX does not fine-tune well if you adhere to that...
I guess there is something wrong under the hood then with the interpretation of the training data in that format...
I used it for fine-tuning meta-llama/Llama-3.1-8B-Instruct

@madroidmaq
Copy link
Contributor

If you check the chat_temlpate configuration in the meta-llama/Llama-3.1-8B-Instruct repository, you may find that the reason could be the requirement for the function return format, which is inconsistent with the OpenAI format. Llama-3.1-8B-Instruct requires a dictionary to be returned.

You have access to the following functions. To call a function, please respond with JSON for a function call.
'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.'

So if you return according to the document's format, some issues may arise (the format in the base model is inconsistent with the fine-tuning data format). When I reviewed the HuggingFace chat_template document again, I found it also highlighted this part:

If you’re familiar with the OpenAI API, you should pay attention to an important difference here - the tool_call is a dict, but in the OpenAI API it’s a JSON string. Passing a string may cause errors or strange model behaviour!

When I checked the mistral-finetune project again, I found that its data usage is consistent with the OpenAI format.

So I think there might not be a strictly correct format here. The key point is to ensure that the format of your fine-tuning dataset needs to be consistent with the base model's format, otherwise problems will arise. I think this part can be explained in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tools example for finetuning seems wrong
4 participants